Symbolic Dynamic Programming for Continuous State and Action MDPs

Qa := ∫ Qa ⊗ P (xj|b,b ′,x, a,y) dxj [Symbolic Substitution] For all bi in Qa Qa := [ Qa ⊗ P (bi|b,x, a,y) ] |b′i=1 ⊕ [ Qa ⊗ P (bi|b,x, a,y) ] |b′i=0 [Case ⊕] Compute final Q-Value (discount and add reward): Qa := R(b,x, a,y)⊕ (γ ⊗Qa) Note that ∫ f (xj)⊗δ[xj−h(z)]dxj = f (xj){xj/h(z)}where the latter operation indicates that any occurrence of xj in f (x ′ j) is symbolically substituted with the case statement h(z) (Sanner, Delgado, de Barros, UAI 2011).